Cost Function Optimization based on Active Learning

Optimization is one of the most important issues in all fields of science and engineering. There are two main categories for optimization problems: continues optimization and discrete optimization. Traditional methods, such as gradient descent, are used for solving continues optimization problems, But for discrete optimization, traditional and many new algorithms are introduced. Due to long time required to solve NPHard problems, special subset of discrete optimization problems, which require non-polynomial time to find exact (global) optimum, some researchers of artificial intelligence suggest heuristic algorithms for solving these hard problems. They model these problems as a state space search in a search graph, where candidate solutions are nodes, and actions specify links between them. Algorithms based on Reinforcement Learning, Simulated Annealing, and Multi Start Local Search are based on heuristic. Other researchers of artificial intelligence have attacked these problems by algorithms inspired by the nature; Evolutionary Algorithms and Ant Colony are in this category [we can also consider Simulated Annealing as an algorithm which mimics the nature]. It is worth noting that Molecular (DNA) Computing has also been used for solving TSP problem. In this project, we consider the problem of optimizing discrete cost functions, especially those of combinatorial optimization problems. Bin-Packing problem, a famous NP-Hard problem, is our base problem, which our algorithm is tested on it. Our approach is based on Reinforcement Learning and Active Learning Method (ALM). Reinforcement Learning is explained well in the literature (Sutton & Barto 1998), and Boyan & Moore have introduced STAGE based on Reinforcement Learning for optimizing hard combinatorial optimization problems (Boyan 1998). ALM is a supervised learning algorithm introduced by Bagheri & Honda (Bagheri & Honda 1999), which tries to learn the behavior of an unknown nonlinear function by using a fuzzy curve fitting method. This algorithm is similar to human brain’s learning process. We try to introduce a new optimization algorithm based on STAGE and ALM. This algorithm uses ALM to automatically extract useful features of the problem and then uses them to guide search process toward lower cost solutions in the search space graph. Up to now, we have analyzed different local search algorithms in STAGE and cost function structure of the bin-packing problem. The result of this research is a paper, which we have submitted to the 19 International Conference on Machine Learning (ICML2002), Sydney, Australia (G. Haffari & S. B. Shouraki 2002). Currently, we are trying to propose our final algorithm, and test it on bin-packing and other benchmark problems. In this report, we review our paper and give a schedule for doing future works. Previous researches have shown the success of using Reinforcement Learning in solving combinatorial optimization problems. The main idea of these methods is to learn (near) optimal evaluation functions to improve local searches and find (near) optimal solutions. STAGE algorithm, introduced by Boyan & Moore, is one of the most important algorithms in this area. In this report, we focus on Bin-Packing problem, an important NP-Complete problem. We analyze cost surface structure of this problem and investigate “big valley” structure for the set of its local minima. The result gives reasons for STAGE’s success in solving this problem. Then by comparing the results of experiments on Bin-Packing problem, we analyze the effectiveness of steepest-descent hill climbing, stochastic hill climbing and first-improvement hill climbing as the local search algorithms in STAGE.